Foundations for non-linear solver and polymorphic application - #15287

Fixes #1317
Fixes #5738
Fixes #12919
(also fixes a FIX comment that is more than 10 years old according to git blame)

Note: although this PR fixes most typical use-cases for type inference against generic functions, it is intentionally incomplete, and it is made in a way to limit implications to small scope.

This PR has essentially three components (better infer, better solve, better apply - all three are needed for this MVP to work):

• A "tiny" change to constraints.py: if the actual function is generic, we unify it with template before inferring constraints. This prevents leaking generic type variables of actual in the solutions (which makes no sense), but also introduces new kind of constraints T <: F[S], where type variables we solve for appear in target type. These are much harder to solve, but also it is a great opportunity to play with them to prepare for single bin inference (if we will switch to it in some form later). Note unifying is not the best solution, but a good first approximation (see below on what is the best solution).
• New more sophisticated constraint solver in solve.py. The full algorithm is outlined in the docstring for solve_non_linear(). It looks like it should be able to solve arbitrary constraints that don't (indirectly) contain "F-bounded" things like T <: list[T]. Very short the idea is to compute transitive closure, then organize constraints by topologically sorted SCCs.
• Polymorphic type argument application in checkexpr.py. In cases where solver identifies there are free variables (e.g. we have just one constraint S <: list[T], so T is free, and solution for S is list[T]) it will apply the solutions while creating new generic functions. For example, if we have a function def [S, T] (fn: Callable[[S], T]) -> Callable[[S], T] applied to a function def [U] (x: U) -> U, this will result in def [T] (T) -> T as the return.

I want to put here some thoughts on the last ingredient, since it may be mysterious, but now it seems to me it is actually a very well defined procedure. The key point here is thinking about generic functions as about infinite intersections or infinite overloads. Now reducing these infinite overloads/intersections to finite ones it is easy to understand what is actually going on. For example, imagine we live in a world with just two types int and str. Now we have two functions:

T = TypeVar("T")
S = TypeVar("S")
U = TypeVar("U")

def dec(fn: Callable[[T], S]) -> Callable[[T], S]: ...
def id(x: U) -> U: ...

the first one can be seen as overload over

((int) -> int) -> ((int) -> int)  # 1
((int) -> str) -> ((int) -> str)  # 2
((str) -> int) -> ((str) -> int)  # 3
((str) -> str) -> ((str) -> str)  # 4

and second as an overload over

(int) -> int
(str) -> str

Now what happens when I apply dec(id)? We need to choose an overload that matches the argument (this is what we call type inference), but here is a trick, in this case two overloads of dec match the argument type. So (and btw I think we are missing this for real overloads) we construct a new overload that returns intersection of matching overloads # 1 and # 4. So if we generalize this intuition to the general case, the inference is selection of an (infinite) parametrized subset among the bigger parameterized set of intersecting types. The only question is whether resulting infinite intersection is representable in our type system. For example forall T. dict[T, T] can make sense but is not representable, while forall T. (T) -> T is a well defined type. And finally, there is a very easy way to find whether a type is representable or not, we are already doing this during semantic analyzis. I use the same logic (that I used to view as ad-hoc because of lack of good syntax for callables) to bind type variables in the inferred type.

OK, so here is the list of missing features, and some comments on them:

1. Instead of unifying the actual with template we should include actual's variables in variable set we solve for, as explained in Generic inference regression from #5699 #5738 (comment). Note however, this will work only together with the next item
2. We need to (iteratively) infer secondary constraints after linear propagation, e.g. Sequence[T] <: S <: Sequence[U] => T <: U
3. Support ParamSpec (and probably TypeVarTuple). Current support for applying callables with ParamSpec to generics is hacky, and kind of dead-end. Although (Callable[P, T]) -> Callable[P, List[T]] works when applied to id, even a slight variation like (Callable[P, List[T]]) -> Callable[P, T] fails. I think it needs to be re-worked in the framework I propose (the tests I added are just to be sure I don't break existing code)
4. Support actual types that are generic in type variables with upper bounds or values (likely we just need to be careful when propagating constraints and choosing free variable within an SCC).
5. Add backtracking for upper/lower bound choice. In general, in the current "Hanoi Tower" inference scheme it is very hard to backtrack, but in in this specific choice in the new solver, it should be totally possible to switch from lower to upper bound on a previous step, if we found no solution (or <nothing>/object).
6. After we polish it, we can use the new solver in more situations, e.g. for return type context, and for unification during callable subtyping.
7. Long term we may want to allow instances to bind type variables, at least for things like LRUCache[[x: T], T]. Btw note that I apply force expansion to type aliases and callback protocols. Since I can't transform e.g. A = Callable[[T], T] into a generic callable without getting proper type.
8. We need to figure out a solution for scenarios where non-linear targets with free variables and constant targets mix without secondary constraints, like T <: List[int], T <: List[S].

I am planning to address at least majority of the above items, but I think we should move slowly, since in my experience type inference is really fragile topic with hard to predict long reaching consequences. Please play with this PR if you want to and have time, and please suggest tests to add.